We will combine information from Census Name data cnames and Florida Census data FLcensus to predict ethnic outcomes for the Florida voters data, in which we already have information on individual self-reported (true) outcome to evaluate our accuracy.
Step 1. First look at the Florida data
We will read the data
cnames<-read.csv("data/cnames.csv")
FLvoters<-read.csv("data/FLvoters.csv")
FLcensus<-read.csv("data/FLCensusVTD.csv")
dim(FLvoters)
[1] 10000 6
Lets examine the structure of the data (Note that this is individual data that contain the joint distribution and of course we can recover everything that we need.)
head(FLvoters)
head(cnames)
head(FLcensus)
Question: What information do we have in each dataset?
\[
\begin{aligned}
& \Pr[\text{Race} \mid \text{Surname}] \\
& \Pr[\text{Surname} \mid \text{Race}] \\
& \Pr[\text{Race} \mid \text{Surname, Residence}] \\
& \Pr[\text{Race} \mid \text{Residence}] \\
\end{aligned}
\]
Step 2. Clean up the data a bit
Preliminary: Matching Data
We will drop those data points that are NOT matched to the census names.
Approach #1: Let’s learn how to select data using match()
x<-c("blue","white","red")
y<-c("black","blue")
# Give me the corresponding position in y
match(x,y)
[1] 2 NA NA
# Give me the corresponding position in x
match(y,x)
[1] NA 1
If I would like to select only x values matched to y, then I will use the following code
!is.na(match(x,y))
[1] TRUE FALSE FALSE
x[!is.na(match(x,y))]
[1] "blue"
Approach #2: Tidy Approach
library(dplyr)
x<-data.frame(x=c("blue","white","red"))
y<-data.frame(y=c("black","blue"))
x %>%
filter(x %in% y$y)
Clean Our Data
Keep only those surnames that can be matched to the Census Data
Approach #1
FLvoters <- FLvoters[!is.na(match(FLvoters$surname,cnames$surname)),]
Approach #2
FLvoters <- FLvoters %>%
filter(surname %in% cnames$surname)
Let’s drop missing values
FLvoters<-na.omit(FLvoters)
dim(FLvoters)
[1] 8022 6
Lets combine the percentage of other races
cnames$pctothers<-100-(cnames$pctapi+cnames$pctblack+cnames$pctwhite+cnames$pcthispanic)
head(cnames)
Step 3: Prediction
\[
\Pr[\text{race} \mid \text{surname}]
\]
- Split the data by races. For example, whites and blacks.
whites <- subset(FLvoters,subset = (race == "white"))
head(whites)
- Match the Florida data and the Census data using surname.
w.indx <- match(whites$surname, cnames$surname)
head(whites)
head(w.indx)
[1] 8610 237 4131 2244 27852 3495
For example, the first person is PIEDRA, and this surname appears as the \(8610^{th}\) person in the Census data.
head(cnames)
cnames[8610,]
- For the sample of whites (census names corresponding to whites), the maximum of conditional probabilities should be the conditional probabilitiy of being white given the surname.
\[
\max\{\text{pctwhite,pctblack,pctapi,pcthispanic}\}=\text{pctwhite}
\]
Example
cnames[8610,]
max(cnames[8610, c("pctwhite","pctblack","pctapi","pctaian","pct2prace","pcthispanic")])
Is it equal to the percentage white? Or, is the percentage of whites the largest number?
max(cnames[8610, c("pctwhite","pctblack","pctapi","pctaian","pct2prace","pcthispanic")]) == cnames$pctwhite[8610]
Then we know we did not predict correctly. Lets do it for every osbervation then.
# Pick our relevant variables, which should sum to 100
vars<-c("pctwhite","pctblack","pctapi","pctaian","pct2prace","pcthispanic")
cnames[w.indx, vars]
#Calculate the maximum and compare it to the cond prob of being
# whites
#apply(cnames[w.indx, vars], 1, max)
head(apply(cnames[w.indx, vars], 1, max) )
comparison<-apply(cnames[w.indx, vars], 1, max) == cnames$pctwhite[w.indx]
head(comparison)
Lets look at how we should use apply() in R. see mv05_cond_indep03_apply.Rmd
- The success rates are defined as the instances when these two are indeed the same.
comparison
mean(comparison)
- We repeat the process (3) and (4) for other races.
# We can repeat the process for other races
# Blacks
blacks<- subset(FLvoters, subset = (race == "black"))
b.indx<-match(blacks$surname,cnames$surname)
mean(apply(cnames[b.indx, vars], 1, max) == cnames$pctblack[b.indx])
# Asians
asians <- subset(FLvoters, subset = (race == "asian"))
a.indx<- match(asians$surname, cnames$surname)
mean(apply(cnames[a.indx, vars], 1, max) == cnames$pctapi[a.indx])
# Hispanics
hispanics <- subset(FLvoters, subset = (race == "hispanic"))
h.indx <- match(hispanics$surname, cnames$surname)
mean(apply(cnames[h.indx, vars], 1, max) == cnames$pcthispanic[h.indx])
Step 4: Prediction taking into account more information
Can we improve the precision by incorporating more information, for example, on residence?
\[
\Pr[\text{race}|\text{surname,residence}]
\]
\[
\Pr[\text{race}|\text{surname,residence}]=\frac{\Pr[\text{surname}|\text{race,residence}]\Pr[\text{race}|\text{residence}]}{\Pr[\text{surname}|\text{residence}]}
\]
What do We have:
\[
\Pr[\text{race}|\text{surname}] \quad \text{(Census data)}
\]
head(cnames)
\[
\Pr[\text{race}|\text{residence}] \text{ and } \Pr[\text{residence}] \quad \text{(Florida Census data)}
\]
head(FLcensus)
We do not have in these two datasets (note that FLvoters is only our validation dataset):
\[
\Pr[\text{surname}\mid \text{race},\text{residence}]
\]
\[
\Pr[\text{surname}\mid \text{residence}]
\]
Note that with the application of the Law of Total Probability
we can obtain the second one
\[
\begin{aligned}
& \Pr[\text{surname}|\text{residence}] \\
= & \sum\Pr[\text{surname}|\text{race,residence}]\cdot\Pr[\text{race}|\text{residence}] \\
\end{aligned}
\] Now the key is to obtain \[
\Pr[\text{surname}\mid \text{race,residence}]
\]
We would invoke our conditional independence assumption
\[
\Pr[\text{surname} \mid \text{race, residence}]=\Pr[\text{surname}\mid \text{race}]
\]
Does it make sense? Or, when will it be violated? Conditional independence implies that once we know a voter’s race, her residence location does not give us any additional information about her surname.
There is NO strong geographical concentration of certain surnames in Florida within a racial categroy.
\[
\begin{aligned}
& \Pr[\text{surname}|\text{race,}\text{residence}] \\
\\
\\
= & \Pr[\text{surname}|\text{race}] \\
\\
\\
= & \frac{\Pr[\text{race}|\text{surname}]\Pr[\text{surname}]}{\Pr[\text{race}]}
\end{aligned}
\] We can obtain these terms:
- Census data: \[\Pr[\text{race}|\text{surname}] \text{ and } \Pr[\text{surname}]\]
head(cnames)
- Florida Census data: \[
\Pr[\text{race}]=\sum_{\text{residence}}\Pr[\text{race}|\text{residence}]\Pr[\text{residence}]
\]
head(FLcensus)
race.prop <- apply(FLcensus[,c("white", "black", "api", "hispanic", "others")],
2,
weighted.mean,
weights = FLCensus$total.pop)
race.prop
So, if we are interested in \[
\Pr[\text{surname} \mid \text{race=white}]=\frac{\Pr[\text{white} \mid \text{surname}]\Pr[\text{surname}]}{\Pr[\text{race=white}]}
\]
head(cnames)
total.count<- sum(cnames$count)
cnames$name.white <- (cnames$pctwhite/100)*(cnames$count/total.count)/race.prop["white"]
head(cnames)
cnames$name.black <- (cnames$pctblack/100)*(cnames$count/total.count)/race.prop["black"]
cnames$name.hispanic <- (cnames$pcthispanic/100)*(cnames$count/total.count)/race.prop["hispanic"]
cnames$name.api <- (cnames$pctapi/100)*(cnames$count/total.count)/race.prop["api"]
cnames$name.others <- (cnames$pctothers/100)*(cnames$count/total.count)/race.prop["others"]
Lets look at what we have now
head(cnames)
Now let’s get the following information
\[
\begin{aligned}
& \Pr[\text{surname}|\text{residence}] \\
= & \sum\Pr[\text{surname}|\text{race,residence}]\cdot\Pr[\text{race}|\text{residence}] \\
= & \sum\Pr[\text{surname}|\text{race}]\cdot\Pr[\text{race}|\text{residence}]
\end{aligned}
\]
Lets merge all the information with our validation dataset
FLvoters<- merge(x=FLvoters, y=FLcensus, by=c("county","VTD"), all= FALSE)
head(FLvoters)
indx <- match(FLvoters$surname, cnames$surname)
FLvoters$name.residence <-cnames$name.white[indx]*FLvoters$white+cnames$name.black[indx]*FLvoters$black+cnames$name.hispanic[indx]*FLvoters$hispanic+cnames$name.api[indx]*FLvoters$api+cnames$name.others[indx]*FLvoters$others
head(FLvoters)
Finally,
$$
\[\begin{aligned}
& \Pr[\text{race}|\text{surname,residence}] \\
\\
= & \frac{\Pr[\text{surname}|\text{race,residence}]\Pr[\text{race}|\text{residence}]}{\Pr[\text{surname}|\text{residence}]} \\
\\
& = \frac{\Pr[\text{surname}|\text{race}]\Pr[\text{race}|\text{residence}]}{\Pr[\text{surname}|\text{residence}]}
\end{aligned}\]
$$
For example, \[
\begin{aligned}
& \Pr[\text{white}|\text{surname,residence}] \\
\\
= & \frac{\Pr[\text{surname}|\text{white}]\Pr[\text{white}|\text{residence}]}{\Pr[\text{surname}|\text{residence}]}
\end{aligned}
\]
FLvoters$predict.white <- cnames$name.white[indx]*FLvoters$white/FLvoters$name.residence
FLvoters$predict.black <- cnames$name.black[indx]*FLvoters$black/FLvoters$name.residence
FLvoters$predict.hispanic <- cnames$name.hispanic[indx]*FLvoters$hispanic/FLvoters$name.residence
FLvoters$predict.api <- cnames$name.api[indx]*FLvoters$api/FLvoters$name.residence
FLvoters$predict.others <- cnames$name.others[indx]*FLvoters$others/FLvoters$name.residence
## relevant variables
vars1 <- c("predict.white", "predict.black", "predict.hispanic", "predict.api", "predict.others")
## whites
whites <- subset(FLvoters, subset = (race == "white"))
mean(apply(whites[, vars1], 1, max) == whites$predict.white)
## blacks
blacks <- subset(FLvoters, subset = (race == "black"))
mean(apply(blacks[, vars1], 1, max) == blacks$predict.black)
LS0tCnRpdGxlOiAiUiBOb3RlYm9vayIKb3V0cHV0OiBodG1sX25vdGVib29rCi0tLQoKV2Ugd2lsbCBjb21iaW5lIGluZm9ybWF0aW9uIGZyb20gQ2Vuc3VzIE5hbWUgZGF0YSBgY25hbWVzYCBhbmQgRmxvcmlkYSBDZW5zdXMgZGF0YSBgRkxjZW5zdXNgIHRvIHByZWRpY3QgZXRobmljIG91dGNvbWVzIGZvciB0aGUgRmxvcmlkYSB2b3RlcnMgZGF0YSwgaW4gd2hpY2ggd2UgYWxyZWFkeSBoYXZlIGluZm9ybWF0aW9uIG9uIGluZGl2aWR1YWwgc2VsZi1yZXBvcnRlZCAodHJ1ZSkgb3V0Y29tZSB0byBldmFsdWF0ZSBvdXIgYWNjdXJhY3kuIAoKIyMgU3RlcCAxLiBGaXJzdCBsb29rIGF0IHRoZSBGbG9yaWRhIGRhdGEKCgpXZSB3aWxsIHJlYWQgdGhlIGRhdGEKCmBgYHtyfQpjbmFtZXM8LXJlYWQuY3N2KCJkYXRhL2NuYW1lcy5jc3YiKQpGTHZvdGVyczwtcmVhZC5jc3YoImRhdGEvRkx2b3RlcnMuY3N2IikKRkxjZW5zdXM8LXJlYWQuY3N2KCJkYXRhL0ZMQ2Vuc3VzVlRELmNzdiIpCmRpbShGTHZvdGVycykKCmBgYAoKCkxldHMgZXhhbWluZSB0aGUgc3RydWN0dXJlIG9mIHRoZSBkYXRhIChOb3RlIHRoYXQgdGhpcyBpcyBpbmRpdmlkdWFsIGRhdGEgdGhhdCBjb250YWluIHRoZSBqb2ludCBkaXN0cmlidXRpb24gYW5kIG9mIGNvdXJzZSB3ZSBjYW4gcmVjb3ZlciBldmVyeXRoaW5nIHRoYXQgd2UgbmVlZC4pCgpgYGB7cn0KaGVhZChGTHZvdGVycykKYGBgCgoKYGBge3J9CmhlYWQoY25hbWVzKQpgYGAKCgpgYGB7cn0KaGVhZChGTGNlbnN1cykKYGBgCgoqKlF1ZXN0aW9uOioqIFdoYXQgaW5mb3JtYXRpb24gZG8gd2UgaGF2ZSBpbiBlYWNoIGRhdGFzZXQ/CgokJApcYmVnaW57YWxpZ25lZH0KJiBcUHJbXHRleHR7UmFjZX0gXG1pZCBcdGV4dHtTdXJuYW1lfV0gXFwKJiBcUHJbXHRleHR7U3VybmFtZX0gXG1pZCBcdGV4dHtSYWNlfV0gXFwKJiBcUHJbXHRleHR7UmFjZX0gXG1pZCBcdGV4dHtTdXJuYW1lLCBSZXNpZGVuY2V9XSBcXAomIFxQcltcdGV4dHtSYWNlfSBcbWlkIFx0ZXh0e1Jlc2lkZW5jZX1dIFxcClxlbmR7YWxpZ25lZH0KJCQKCiMjIFN0ZXAgMi4gQ2xlYW4gdXAgdGhlIGRhdGEgYSBiaXQKCiMjIyBQcmVsaW1pbmFyeTogTWF0Y2hpbmcgRGF0YQoKV2Ugd2lsbCBkcm9wIHRob3NlIGRhdGEgcG9pbnRzIHRoYXQgYXJlIE5PVCBtYXRjaGVkIHRvIHRoZSBjZW5zdXMgbmFtZXMuIAoKKipBcHByb2FjaCAjMToqKiBMZXQncyBsZWFybiBob3cgdG8gc2VsZWN0IGRhdGEgdXNpbmcgYG1hdGNoKClgCgpgYGB7cn0KeDwtYygiYmx1ZSIsIndoaXRlIiwicmVkIikKeTwtYygiYmxhY2siLCJibHVlIikKCiMgR2l2ZSBtZSB0aGUgY29ycmVzcG9uZGluZyBwb3NpdGlvbiBpbiB5Cm1hdGNoKHgseSkKCiMgR2l2ZSBtZSB0aGUgY29ycmVzcG9uZGluZyBwb3NpdGlvbiBpbiB4Cm1hdGNoKHkseCkKYGBgCgpJZiBJIHdvdWxkIGxpa2UgdG8gc2VsZWN0IG9ubHkgeCB2YWx1ZXMgbWF0Y2hlZCB0byB5LCB0aGVuIEkgd2lsbCB1c2UgdGhlIGZvbGxvd2luZyBjb2RlCgpgYGB7cn0KIWlzLm5hKG1hdGNoKHgseSkpCnhbIWlzLm5hKG1hdGNoKHgseSkpXQpgYGAKCioqQXBwcm9hY2ggIzI6IFRpZHkgQXBwcm9hY2gqKgoKYGBge3J9CmxpYnJhcnkoZHBseXIpCng8LWRhdGEuZnJhbWUoeD1jKCJibHVlIiwid2hpdGUiLCJyZWQiKSkKeTwtZGF0YS5mcmFtZSh5PWMoImJsYWNrIiwiYmx1ZSIpKQoKeCAlPiUgCiAgZmlsdGVyKHggJWluJSB5JHkpCmBgYAoKIyMjIENsZWFuIE91ciBEYXRhCgpLZWVwIG9ubHkgdGhvc2Ugc3VybmFtZXMgdGhhdCBjYW4gYmUgbWF0Y2hlZCB0byB0aGUgQ2Vuc3VzIERhdGEKCioqQXBwcm9hY2ggIzEqKgoKYGBge3J9CkZMdm90ZXJzIDwtIEZMdm90ZXJzWyFpcy5uYShtYXRjaChGTHZvdGVycyRzdXJuYW1lLGNuYW1lcyRzdXJuYW1lKSksXQpgYGAKCioqQXBwcm9hY2ggIzIqKgoKYGBge3J9CkZMdm90ZXJzIDwtIEZMdm90ZXJzICU+JSAKICBmaWx0ZXIoc3VybmFtZSAlaW4lIGNuYW1lcyRzdXJuYW1lKQpgYGAKCgpMZXQncyBkcm9wIG1pc3NpbmcgdmFsdWVzCgpgYGB7cn0KRkx2b3RlcnM8LW5hLm9taXQoRkx2b3RlcnMpCmRpbShGTHZvdGVycykKYGBgCgoKTGV0cyBjb21iaW5lIHRoZSBwZXJjZW50YWdlIG9mIG90aGVyIHJhY2VzCgpgYGB7cn0KY25hbWVzJHBjdG90aGVyczwtMTAwLShjbmFtZXMkcGN0YXBpK2NuYW1lcyRwY3RibGFjaytjbmFtZXMkcGN0d2hpdGUrY25hbWVzJHBjdGhpc3BhbmljKQpoZWFkKGNuYW1lcykKYGBgCgojIyBTdGVwIDM6IFByZWRpY3Rpb24KCiQkClxQcltcdGV4dHtyYWNlfSBcbWlkIFx0ZXh0e3N1cm5hbWV9XQokJAoKMS4gU3BsaXQgdGhlIGRhdGEgYnkgcmFjZXMuIEZvciBleGFtcGxlLCB3aGl0ZXMgYW5kIGJsYWNrcy4gCgpgYGB7cn0Kd2hpdGVzIDwtIHN1YnNldChGTHZvdGVycyxzdWJzZXQgPSAocmFjZSA9PSAid2hpdGUiKSkgCmhlYWQod2hpdGVzKQpgYGAKCgoyLiBNYXRjaCB0aGUgRmxvcmlkYSBkYXRhIGFuZCB0aGUgQ2Vuc3VzIGRhdGEgdXNpbmcgc3VybmFtZS4KCmBgYHtyfQp3LmluZHggPC0gbWF0Y2god2hpdGVzJHN1cm5hbWUsIGNuYW1lcyRzdXJuYW1lKQpoZWFkKHdoaXRlcykKaGVhZCh3LmluZHgpCmBgYAoKCkZvciBleGFtcGxlLCB0aGUgZmlyc3QgcGVyc29uIGlzIFBJRURSQSwgYW5kIHRoaXMgc3VybmFtZSBhcHBlYXJzIGFzIHRoZSAkODYxMF57dGh9JCBwZXJzb24gaW4gdGhlIENlbnN1cyBkYXRhLgoKYGBge3J9CmhlYWQoY25hbWVzKQpgYGAKCgpgYGB7cn0KY25hbWVzWzg2MTAsXQpgYGAKCgoKMy4gRm9yIHRoZSBzYW1wbGUgb2Ygd2hpdGVzIChjZW5zdXMgbmFtZXMgY29ycmVzcG9uZGluZyB0byB3aGl0ZXMpLCB0aGUgbWF4aW11bSBvZiBjb25kaXRpb25hbCBwcm9iYWJpbGl0aWVzIHNob3VsZCBiZSB0aGUgY29uZGl0aW9uYWwgcHJvYmFiaWxpdGl5IG9mIGJlaW5nIHdoaXRlIGdpdmVuIHRoZSBzdXJuYW1lLgoKJCQKXG1heFx7XHRleHR7cGN0d2hpdGUscGN0YmxhY2sscGN0YXBpLHBjdGhpc3BhbmljfVx9PVx0ZXh0e3BjdHdoaXRlfQokJAoKRXhhbXBsZQoKYGBge3J9CmNuYW1lc1s4NjEwLF0KYGBgCgpgYGB7cn0KbWF4KGNuYW1lc1s4NjEwLCBjKCJwY3R3aGl0ZSIsInBjdGJsYWNrIiwicGN0YXBpIiwicGN0YWlhbiIsInBjdDJwcmFjZSIsInBjdGhpc3BhbmljIildKQpgYGAKCklzIGl0IGVxdWFsIHRvIHRoZSBwZXJjZW50YWdlIHdoaXRlPyBPciwgaXMgdGhlIHBlcmNlbnRhZ2Ugb2Ygd2hpdGVzIHRoZSBsYXJnZXN0IG51bWJlcj8gCgpgYGB7cn0KbWF4KGNuYW1lc1s4NjEwLCBjKCJwY3R3aGl0ZSIsInBjdGJsYWNrIiwicGN0YXBpIiwicGN0YWlhbiIsInBjdDJwcmFjZSIsInBjdGhpc3BhbmljIildKSA9PSBjbmFtZXMkcGN0d2hpdGVbODYxMF0KYGBgCgpUaGVuIHdlIGtub3cgd2UgZGlkIG5vdCBwcmVkaWN0IGNvcnJlY3RseS4gTGV0cyBkbyBpdCBmb3IgZXZlcnkgb3NiZXJ2YXRpb24gdGhlbi4gCgpgYGB7cn0KIyBQaWNrIG91ciByZWxldmFudCB2YXJpYWJsZXMsIHdoaWNoIHNob3VsZCBzdW0gdG8gMTAwIAp2YXJzPC1jKCJwY3R3aGl0ZSIsInBjdGJsYWNrIiwicGN0YXBpIiwicGN0YWlhbiIsInBjdDJwcmFjZSIsInBjdGhpc3BhbmljIikKCmNuYW1lc1t3LmluZHgsIHZhcnNdCmBgYAoKCmBgYHtyfQoKCiNDYWxjdWxhdGUgdGhlIG1heGltdW0gYW5kIGNvbXBhcmUgaXQgdG8gdGhlIGNvbmQgcHJvYiBvZiBiZWluZwojIHdoaXRlcwoKI2FwcGx5KGNuYW1lc1t3LmluZHgsIHZhcnNdLCAxLCBtYXgpCgpoZWFkKGFwcGx5KGNuYW1lc1t3LmluZHgsIHZhcnNdLCAxLCBtYXgpICkKCmNvbXBhcmlzb248LWFwcGx5KGNuYW1lc1t3LmluZHgsIHZhcnNdLCAxLCBtYXgpID09IGNuYW1lcyRwY3R3aGl0ZVt3LmluZHhdCmhlYWQoY29tcGFyaXNvbikKCmBgYAoKCgpMZXRzIGxvb2sgYXQgaG93IHdlIHNob3VsZCB1c2UgYGFwcGx5KClgIGluIFIuIHNlZSBtdjA1X2NvbmRfaW5kZXAwM19hcHBseS5SbWQKCgoKCjQuIFRoZSBzdWNjZXNzIHJhdGVzIGFyZSBkZWZpbmVkIGFzIHRoZSBpbnN0YW5jZXMgd2hlbiB0aGVzZSB0d28gYXJlIGluZGVlZCB0aGUgc2FtZS4gCgpgYGB7cn0KY29tcGFyaXNvbgpgYGAKCgpgYGB7cn0KbWVhbihjb21wYXJpc29uKQpgYGAKCgo1LiBXZSByZXBlYXQgdGhlIHByb2Nlc3MgKDMpIGFuZCAoNCkgZm9yIG90aGVyIHJhY2VzLiAKCgpgYGB7cn0KIyBXZSBjYW4gcmVwZWF0IHRoZSBwcm9jZXNzIGZvciBvdGhlciByYWNlcwojIEJsYWNrcyAKCmJsYWNrczwtIHN1YnNldChGTHZvdGVycywgc3Vic2V0ID0gKHJhY2UgPT0gImJsYWNrIikpIApiLmluZHg8LW1hdGNoKGJsYWNrcyRzdXJuYW1lLGNuYW1lcyRzdXJuYW1lKSAKbWVhbihhcHBseShjbmFtZXNbYi5pbmR4LCB2YXJzXSwgMSwgbWF4KSA9PSBjbmFtZXMkcGN0YmxhY2tbYi5pbmR4XSkKCmBgYAoKCmBgYHtyfQojIEFzaWFucyAKYXNpYW5zIDwtIHN1YnNldChGTHZvdGVycywgc3Vic2V0ID0gKHJhY2UgPT0gImFzaWFuIikpIAphLmluZHg8LSBtYXRjaChhc2lhbnMkc3VybmFtZSwgY25hbWVzJHN1cm5hbWUpIAptZWFuKGFwcGx5KGNuYW1lc1thLmluZHgsIHZhcnNdLCAxLCBtYXgpID09IGNuYW1lcyRwY3RhcGlbYS5pbmR4XSkKCgpgYGAKCgpgYGB7cn0KIyBIaXNwYW5pY3MgCmhpc3BhbmljcyA8LSBzdWJzZXQoRkx2b3RlcnMsIHN1YnNldCA9IChyYWNlID09ICJoaXNwYW5pYyIpKSAKaC5pbmR4IDwtIG1hdGNoKGhpc3BhbmljcyRzdXJuYW1lLCBjbmFtZXMkc3VybmFtZSkgCm1lYW4oYXBwbHkoY25hbWVzW2guaW5keCwgdmFyc10sIDEsIG1heCkgPT0gY25hbWVzJHBjdGhpc3BhbmljW2guaW5keF0pCmBgYAoKCgojIyBTdGVwIDQ6IFByZWRpY3Rpb24gdGFraW5nIGludG8gYWNjb3VudCBtb3JlIGluZm9ybWF0aW9uCgpDYW4gd2UgaW1wcm92ZSB0aGUgcHJlY2lzaW9uIGJ5IGluY29ycG9yYXRpbmcgbW9yZSBpbmZvcm1hdGlvbiwgZm9yIGV4YW1wbGUsIG9uIHJlc2lkZW5jZT8KCiQkCiBcUHJbXHRleHR7cmFjZX18XHRleHR7c3VybmFtZSxyZXNpZGVuY2V9XQokJAoKCgoKJCQKXFByW1x0ZXh0e3JhY2V9fFx0ZXh0e3N1cm5hbWUscmVzaWRlbmNlfV09XGZyYWN7XFByW1x0ZXh0e3N1cm5hbWV9fFx0ZXh0e3JhY2UscmVzaWRlbmNlfV1cUHJbXHRleHR7cmFjZX18XHRleHR7cmVzaWRlbmNlfV19e1xQcltcdGV4dHtzdXJuYW1lfXxcdGV4dHtyZXNpZGVuY2V9XX0KJCQKCgpXaGF0IGRvIFdlIGhhdmU6CgokJApcUHJbXHRleHR7cmFjZX18XHRleHR7c3VybmFtZX1dIFxxdWFkIFx0ZXh0eyhDZW5zdXMgZGF0YSl9CiQkCgpgYGB7cn0KaGVhZChjbmFtZXMpCmBgYAoKCiQkClxQcltcdGV4dHtyYWNlfXxcdGV4dHtyZXNpZGVuY2V9XSBcdGV4dHsgYW5kIH0gXFByW1x0ZXh0e3Jlc2lkZW5jZX1dIFxxdWFkIFx0ZXh0eyhGbG9yaWRhIENlbnN1cyBkYXRhKX0KJCQKYGBge3J9CmhlYWQoRkxjZW5zdXMpCmBgYAoKV2UgZG8gbm90IGhhdmUgaW4gdGhlc2UgdHdvIGRhdGFzZXRzIChub3RlIHRoYXQgYEZMdm90ZXJzYCBpcyBvbmx5IG91ciB2YWxpZGF0aW9uIGRhdGFzZXQpOgoKCiQkClxQcltcdGV4dHtzdXJuYW1lfVxtaWQgXHRleHR7cmFjZX0sXHRleHR7cmVzaWRlbmNlfV0gCiQkCgokJApcUHJbXHRleHR7c3VybmFtZX1cbWlkIFx0ZXh0e3Jlc2lkZW5jZX1dIAokJAoKTm90ZSB0aGF0IHdpdGggdGhlIGFwcGxpY2F0aW9uIG9mIHRoZSBMYXcgb2YgVG90YWwgUHJvYmFiaWxpdHkKCndlIGNhbiBvYnRhaW4gdGhlIHNlY29uZCBvbmUgCgokJApcYmVnaW57YWxpZ25lZH0KJiBcUHJbXHRleHR7c3VybmFtZX18XHRleHR7cmVzaWRlbmNlfV0JXFwKPSAmIFxzdW1cUHJbXHRleHR7c3VybmFtZX18XHRleHR7cmFjZSxyZXNpZGVuY2V9XVxjZG90XFByW1x0ZXh0e3JhY2V9fFx0ZXh0e3Jlc2lkZW5jZX1dIFxcClxlbmR7YWxpZ25lZH0KJCQKTm93IHRoZSBrZXkgaXMgdG8gb2J0YWluIAokJApcUHJbXHRleHR7c3VybmFtZX1cbWlkIFx0ZXh0e3JhY2UscmVzaWRlbmNlfV0gCiQkCgpXZSB3b3VsZCBpbnZva2Ugb3VyIGNvbmRpdGlvbmFsIGluZGVwZW5kZW5jZSBhc3N1bXB0aW9uCgokJApcUHJbXHRleHR7c3VybmFtZX0gXG1pZCBcdGV4dHtyYWNlLCByZXNpZGVuY2V9XT1cUHJbXHRleHR7c3VybmFtZX1cbWlkIFx0ZXh0e3JhY2V9XQokJAoKRG9lcyBpdCBtYWtlIHNlbnNlPyBPciwgd2hlbiB3aWxsIGl0IGJlIHZpb2xhdGVkPyBDb25kaXRpb25hbCBpbmRlcGVuZGVuY2UgaW1wbGllcyB0aGF0IG9uY2Ugd2Uga25vdyBhIHZvdGVyJ3MgcmFjZSwgaGVyIHJlc2lkZW5jZSBsb2NhdGlvbiBkb2VzIG5vdCBnaXZlIHVzIGFueSBhZGRpdGlvbmFsIGluZm9ybWF0aW9uIGFib3V0IGhlciBzdXJuYW1lLiAKClRoZXJlIGlzIE5PIHN0cm9uZyBnZW9ncmFwaGljYWwgY29uY2VudHJhdGlvbiBvZiBjZXJ0YWluIHN1cm5hbWVzIGluIEZsb3JpZGEgd2l0aGluIGEgcmFjaWFsIGNhdGVncm95LiAKCiQkClxiZWdpbnthbGlnbmVkfQomIFxQcltcdGV4dHtzdXJuYW1lfXxcdGV4dHtyYWNlLH1cdGV4dHtyZXNpZGVuY2V9XQlcXApcXApcXAo9ICYgXFByW1x0ZXh0e3N1cm5hbWV9fFx0ZXh0e3JhY2V9XSBcXApcXApcXAo9ICYgXGZyYWN7XFByW1x0ZXh0e3JhY2V9fFx0ZXh0e3N1cm5hbWV9XVxQcltcdGV4dHtzdXJuYW1lfV19e1xQcltcdGV4dHtyYWNlfV19ClxlbmR7YWxpZ25lZH0KJCQKV2UgY2FuIG9idGFpbiB0aGVzZSB0ZXJtczoKCjEuIENlbnN1cyBkYXRhOiAKJCRcUHJbXHRleHR7cmFjZX18XHRleHR7c3VybmFtZX1dIFx0ZXh0eyBhbmQgfSBcUHJbXHRleHR7c3VybmFtZX1dJCQgCgpgYGB7cn0KaGVhZChjbmFtZXMpCmBgYAoKCjIuIEZsb3JpZGEgQ2Vuc3VzIGRhdGE6CiQkClxQcltcdGV4dHtyYWNlfV09XHN1bV97XHRleHR7cmVzaWRlbmNlfX1cUHJbXHRleHR7cmFjZX18XHRleHR7cmVzaWRlbmNlfV1cUHJbXHRleHR7cmVzaWRlbmNlfV0KJCQKYGBge3J9CmhlYWQoRkxjZW5zdXMpCmBgYAoKCmBgYHtyfQpyYWNlLnByb3AgPC0gYXBwbHkoRkxjZW5zdXNbLGMoIndoaXRlIiwgImJsYWNrIiwgImFwaSIsICJoaXNwYW5pYyIsICJvdGhlcnMiKV0sIAoJCQkJCTIsIAoJCQkJCXdlaWdodGVkLm1lYW4sIAoJCQkJCXdlaWdodHMgPSBGTENlbnN1cyR0b3RhbC5wb3ApCnJhY2UucHJvcApgYGAKCgpTbywgaWYgd2UgYXJlIGludGVyZXN0ZWQgaW4gCiQkClxQcltcdGV4dHtzdXJuYW1lfSBcbWlkIFx0ZXh0e3JhY2U9d2hpdGV9XT1cZnJhY3tcUHJbXHRleHR7d2hpdGV9IFxtaWQgXHRleHR7c3VybmFtZX1dXFByW1x0ZXh0e3N1cm5hbWV9XX17XFByW1x0ZXh0e3JhY2U9d2hpdGV9XX0KJCQKCmBgYHtyfQpoZWFkKGNuYW1lcykKdG90YWwuY291bnQ8LSBzdW0oY25hbWVzJGNvdW50KQoKY25hbWVzJG5hbWUud2hpdGUgPC0gKGNuYW1lcyRwY3R3aGl0ZS8xMDApKihjbmFtZXMkY291bnQvdG90YWwuY291bnQpL3JhY2UucHJvcFsid2hpdGUiXQpgYGAKCgpgYGB7cn0KaGVhZChjbmFtZXMpCmBgYAoKCmBgYHtyfQpjbmFtZXMkbmFtZS5ibGFjayA8LSAoY25hbWVzJHBjdGJsYWNrLzEwMCkqKGNuYW1lcyRjb3VudC90b3RhbC5jb3VudCkvcmFjZS5wcm9wWyJibGFjayJdCgpjbmFtZXMkbmFtZS5oaXNwYW5pYyA8LSAoY25hbWVzJHBjdGhpc3BhbmljLzEwMCkqKGNuYW1lcyRjb3VudC90b3RhbC5jb3VudCkvcmFjZS5wcm9wWyJoaXNwYW5pYyJdCgpjbmFtZXMkbmFtZS5hcGkgPC0gKGNuYW1lcyRwY3RhcGkvMTAwKSooY25hbWVzJGNvdW50L3RvdGFsLmNvdW50KS9yYWNlLnByb3BbImFwaSJdCgpjbmFtZXMkbmFtZS5vdGhlcnMgPC0gKGNuYW1lcyRwY3RvdGhlcnMvMTAwKSooY25hbWVzJGNvdW50L3RvdGFsLmNvdW50KS9yYWNlLnByb3BbIm90aGVycyJdCgpgYGAKCkxldHMgbG9vayBhdCB3aGF0IHdlIGhhdmUgbm93CgpgYGB7cn0KaGVhZChjbmFtZXMpCmBgYAoKTm93IGxldCdzIGdldCB0aGUgZm9sbG93aW5nIGluZm9ybWF0aW9uCgokJApcYmVnaW57YWxpZ25lZH0KJiBcUHJbXHRleHR7c3VybmFtZX18XHRleHR7cmVzaWRlbmNlfV0JXFwKPSAmIFxzdW1cUHJbXHRleHR7c3VybmFtZX18XHRleHR7cmFjZSxyZXNpZGVuY2V9XVxjZG90XFByW1x0ZXh0e3JhY2V9fFx0ZXh0e3Jlc2lkZW5jZX1dIFxcCj0gJiBcc3VtXFByW1x0ZXh0e3N1cm5hbWV9fFx0ZXh0e3JhY2V9XVxjZG90XFByW1x0ZXh0e3JhY2V9fFx0ZXh0e3Jlc2lkZW5jZX1dIApcZW5ke2FsaWduZWR9IAokJAoKTGV0cyBtZXJnZSBhbGwgdGhlIGluZm9ybWF0aW9uIHdpdGggb3VyIHZhbGlkYXRpb24gZGF0YXNldAoKYGBge3J9CkZMdm90ZXJzPC0gbWVyZ2UoeD1GTHZvdGVycywgeT1GTGNlbnN1cywgYnk9YygiY291bnR5IiwiVlREIiksIGFsbD0gRkFMU0UpCgpoZWFkKEZMdm90ZXJzKQpgYGAKCgpgYGB7cn0KaW5keCA8LSBtYXRjaChGTHZvdGVycyRzdXJuYW1lLCBjbmFtZXMkc3VybmFtZSkgCgpGTHZvdGVycyRuYW1lLnJlc2lkZW5jZSA8LWNuYW1lcyRuYW1lLndoaXRlW2luZHhdKkZMdm90ZXJzJHdoaXRlK2NuYW1lcyRuYW1lLmJsYWNrW2luZHhdKkZMdm90ZXJzJGJsYWNrK2NuYW1lcyRuYW1lLmhpc3BhbmljW2luZHhdKkZMdm90ZXJzJGhpc3BhbmljK2NuYW1lcyRuYW1lLmFwaVtpbmR4XSpGTHZvdGVycyRhcGkrY25hbWVzJG5hbWUub3RoZXJzW2luZHhdKkZMdm90ZXJzJG90aGVycyAKaGVhZChGTHZvdGVycykKYGBgCgoKRmluYWxseSwKCiQkClxiZWdpbnthbGlnbmVkfQogJiBcUHJbXHRleHR7cmFjZX18XHRleHR7c3VybmFtZSxyZXNpZGVuY2V9XSBcXAkKXFwKID0gJiBcZnJhY3tcUHJbXHRleHR7c3VybmFtZX18XHRleHR7cmFjZSxyZXNpZGVuY2V9XVxQcltcdGV4dHtyYWNlfXxcdGV4dHtyZXNpZGVuY2V9XX17XFByW1x0ZXh0e3N1cm5hbWV9fFx0ZXh0e3Jlc2lkZW5jZX1dfSBcXAogXFwKJgk9IFxmcmFje1xQcltcdGV4dHtzdXJuYW1lfXxcdGV4dHtyYWNlfV1cUHJbXHRleHR7cmFjZX18XHRleHR7cmVzaWRlbmNlfV19e1xQcltcdGV4dHtzdXJuYW1lfXxcdGV4dHtyZXNpZGVuY2V9XX0KClxlbmR7YWxpZ25lZH0KJCQKCkZvciBleGFtcGxlLCAKJCQKXGJlZ2lue2FsaWduZWR9CiYgXFByW1x0ZXh0e3doaXRlfXxcdGV4dHtzdXJuYW1lLHJlc2lkZW5jZX1dIFxcClxcCj0gJiBcZnJhY3tcUHJbXHRleHR7c3VybmFtZX18XHRleHR7d2hpdGV9XVxQcltcdGV4dHt3aGl0ZX18XHRleHR7cmVzaWRlbmNlfV19e1xQcltcdGV4dHtzdXJuYW1lfXxcdGV4dHtyZXNpZGVuY2V9XX0KXGVuZHthbGlnbmVkfQokJAoKYGBge3J9CkZMdm90ZXJzJHByZWRpY3Qud2hpdGUgPC0gY25hbWVzJG5hbWUud2hpdGVbaW5keF0qRkx2b3RlcnMkd2hpdGUvRkx2b3RlcnMkbmFtZS5yZXNpZGVuY2UKRkx2b3RlcnMkcHJlZGljdC5ibGFjayA8LSBjbmFtZXMkbmFtZS5ibGFja1tpbmR4XSpGTHZvdGVycyRibGFjay9GTHZvdGVycyRuYW1lLnJlc2lkZW5jZQpGTHZvdGVycyRwcmVkaWN0Lmhpc3BhbmljIDwtIGNuYW1lcyRuYW1lLmhpc3BhbmljW2luZHhdKkZMdm90ZXJzJGhpc3BhbmljL0ZMdm90ZXJzJG5hbWUucmVzaWRlbmNlCkZMdm90ZXJzJHByZWRpY3QuYXBpIDwtIGNuYW1lcyRuYW1lLmFwaVtpbmR4XSpGTHZvdGVycyRhcGkvRkx2b3RlcnMkbmFtZS5yZXNpZGVuY2UKRkx2b3RlcnMkcHJlZGljdC5vdGhlcnMgPC0gY25hbWVzJG5hbWUub3RoZXJzW2luZHhdKkZMdm90ZXJzJG90aGVycy9GTHZvdGVycyRuYW1lLnJlc2lkZW5jZQpgYGAKCmBgYHtyfQojIyByZWxldmFudCB2YXJpYWJsZXMgCnZhcnMxIDwtIGMoInByZWRpY3Qud2hpdGUiLCAicHJlZGljdC5ibGFjayIsICJwcmVkaWN0Lmhpc3BhbmljIiwgInByZWRpY3QuYXBpIiwgICAgICAgICAgICAicHJlZGljdC5vdGhlcnMiKSAKCiMjIHdoaXRlcyAKd2hpdGVzIDwtIHN1YnNldChGTHZvdGVycywgc3Vic2V0ID0gKHJhY2UgPT0gIndoaXRlIikpIAptZWFuKGFwcGx5KHdoaXRlc1ssIHZhcnMxXSwgMSwgbWF4KSA9PSB3aGl0ZXMkcHJlZGljdC53aGl0ZSkgCgojIyBibGFja3MgCmJsYWNrcyA8LSBzdWJzZXQoRkx2b3RlcnMsIHN1YnNldCA9IChyYWNlID09ICJibGFjayIpKSAKbWVhbihhcHBseShibGFja3NbLCB2YXJzMV0sIDEsIG1heCkgPT0gYmxhY2tzJHByZWRpY3QuYmxhY2spCmBgYAoKCg==